Cyber-Physical Systems Enabled By Unmanned Aerial System-Based Personal Remote Sensing: Data Mission Quality-Centric Design Architectures

In the coming 20 years, unmanned aerial data collection will be of great importance to many sectors of civilian life. Of these systems, Personal Remote Sensing (PRS) Small Unmanned Aerial Systems (sUASs), which are designed for scientic data collection, will need special attention due to their low cost and high value for farming, scientic, and search-andrescue uses, among countless others. Cyber-Physical Systems (CPSs: large-scale, pervasive automated systems that tightly couple sensing and actuation through technology and the environment) can use sUASs as sensors and actuators, leading to even greater possibilities for benet from sUASs. However, this nascent robotic technology presents as many problems as possibilities due to the challenges surrounding the abilities of these systems to perform safely and eectively for personal, academic, and business use. For these systems, whose missions are dened by the data they are sent to collect, safe and reliable mission quality is of highest importance. Much like the dawning of civil manned aviation, civilian sUAS ights demand privacy, accountability, and other ethical factors for societal integration, while safety of the civilian National Airspace (NAS) is always of utmost importance. While the growing popularity of this technology will drive a great effort to integrate sUASs into the NAS, the only long-term solution to this integration problem is one of proper architecture. In this research, a set of architectural requirements for this integration is presented: the Architecture for Ethical Aerial Information Sensing or AERIS. AERIS provides a cohesive set of requirements for any architecture or set of architectures designed for safe, ethical, accurate aerial data collection. In addition to an overview and showcase of possibilities for sUAS-enabled CPSs, specific examples of AERIS-compatible sUAS architectures using various aerospace design methods are shown. Technical contributions include specic improvements to sUAS payload architecture and control software, inertial navigation and complementary lters, and online energy and health state estimation for lithium-polymer batteries in sUAS missions. Several existing sUASs are proled for their ability to comply with AERIS, and the possibilities of AERIS data-driven missions overall is addressed

A Practitioner’s Guide to Small Unmanned Aerial Systems for Bridge Inspection

Small unmanned aerial system(s) (sUAS) are rapidly emerging as a practical means of performing bridge inspections. Under the right condition, sUAS assisted inspections can be safer, faster, and less costly than manned inspections. Many Departments of Transportation in the United States are in the early stages of adopting this emerging technology. However, definitive guidelines for the selection of equipment for various types of bridge inspections or for the possible challenges during sUAS assisted inspections are absent. Given the large investments of time and capital associated with deploying a sUAS assisted bridge inspection program, a synthesis of authors experiences will be useful for technology transfer between academics and practitioners. In this paper, the authors list the challenges associated with sUAS assisted bridge inspection, discuss equipment and technology options suitable for mitigating these challenges, and present case studies for the application of sUAS to several specific bridge inspection scenarios. The authors provide information to sUAS designers and manufacturers who may be unaware of the specific challenges associated with sUAS assisted bridge inspection. As such, the information presented here may reveal the demands in the design of purpose-built sUAS inspection platforms

Fatigue Crack Detection Using Unmanned Aerial Systems in Under-Bridge Inspection

The applications of Unmanned Aerial Systems (UAS) for bridge inspection, with emphasis on under-bridge inspection and fatigue crack detection, were studied in this report. The potential benefits and challenges of using UAS for bridge inspection were identified through an extensive literature survey. The feasibility of using UAS for fatigue crack detection was studied by determining the minimum lighting, camera distance and environmental requirements for three UAS. The DJI Mavic UAS performed better than the others in both indoor and outdoor GPS-denied inspections. An in-service bridge in Ashton, Idaho was inspected using this UAS to find fatigue cracks. No fatigue crack, known or new, was detected in the UAS images but marker lines around the known fatigue cracks (drawn by the inspectors in previous inspections), concrete defects, and steel rust were detected. Thermography showed promising results for fatigue crack detection in a lab setting, but it was not feasible for UAS applications since it had to be performed using active thermography techniques to obtain adequate results. Additionally, image processing algorithms for autonomous detection of both concrete and fatigue cracks were successfully developed. These algorithms, especially for fatigue crack detection, require more images to perform better, but were demonstrated as feasible to aid in a real-time inspection

Estimation of Soil Moisture at Different Soil Levels Using Machine Learning Techniques and Unmanned Aerial Vehicle (UAV) Multispectral Imagery

Soil moisture is a key component of water balance models. Physically, it is a nonlinear function of parameters that are not easily measured spatially, such as soil texture and soil type. Thus, several studies have been conducted on the estimation of soil moisture using remotely sensed data and data mining techniques such as artificial neural networks (ANNs) and support vector machines (SVMs). However, all models developed based on these techniques are limited to site-specific applications where they are trained and their parameters are tuned. Moreover, since the system of non-linear equations produced by and conducted in the machine learning process are not accessible to researchers, each application of these machine learning approaches must repeat these training steps for any new study area. The fact that the results of this machine learning, black box approach cannot be easily transferred to different locations for extraction of soil moisture estimates is frustrating, and it can lead to inaccurate comparisons between methods or model performance. To overcome the Black-box issue, this study employed a powerful technique called genetic programming (GP), which is a combination of an evolutionary algorithm and artificial intelligence, to simulate soil moisture at different levels using high-resolution, multispectral imagery acquired with an unmanned aerial vehicle (UAV). The output of this approach is either a linear or nonlinear empirical equation that can be used by others. The performance of GP was compared with ANN and SVM modeling results. Several sets of high-resolution aerial imagery captured by the Utah State University AggieAir UAV system over two experimental pasture sites located in northern and southern Utah were used for this soil moisture estimation approach. The inputs used to train these models include the reflectance for the visible, near-infrared (NIR), and thermal bands. The results show (1) the performance of GP versus ANN and SVM and (2) the master equation provided by GP, which can be used in other locations and applications

Assessment of Different Methods for Shadow Detection in High-Resolution Optical Imagery and Evaluation of Shadow Impact on Calculation of NDVI and Evapotranspiration

Significant efforts have been made recently in the application of high-resolution remote sensing imagery (i.e., sub-meter) captured by unmanned aerial vehicles (UAVs) for precision agricultural applications for high-value crops such as wine grapes. However, at such high resolution, shadows will appear in the optical imagery effectively reducing the reflectance and emission signal received by imaging sensors. To date, research that evaluates procedures to identify the occurrence of shadows in imagery produced by UAVs is limited. In this study, the performance of four different shadow detection methods used in satellite imagery was evaluated for high-resolution UAV imagery collected over a California vineyard during the Grape Remote sensing Atmospheric Profile and Evapotranspiration eXperiment (GRAPEX) field campaigns. The performance of the shadow detection methods was compared and impacts of shadowed areas on the normalized difference vegetation index (NDVI) and estimated evapotranspiration (ET) using the Two-Source Energy Balance (TSEB) model are presented. The results indicated that two of the shadow detection methods, the supervised classification and index-based methods, had better performance than two other methods. Furthermore, assessment of shadowed pixels in the vine canopy led to significant differences in the calculated NDVI and ET in areas affected by shadows in the high-resolution imagery. Shadows are shown to have the greatest impact on modeled soil heat flux, while net radiation and sensible heat flux are less affected. Shadows also have an impact on the modeled Bowen ratio (ratio of sensible to latent heat) which can be used as an indicator of vine stress level

Assessment of different methods for shadow detection in high-resolution optical imagery and evaluation of shadow impact on calculation of NDVI, and evapotranspiration

Significant efforts have been made recently in the application of high-resolution remote sensing imagery (i.e., sub-meter) captured by unmanned aerial vehicles (UAVs) for precision agricultural applications for high-value crops such as wine grapes. However, at such high resolution, shadows will appear in the optical imagery effectively reducing the reflectance and emission signal received by imaging sensors. To date, research that evaluates procedures to identify the occurrence of shadows in imagery produced by UAVs is limited. In this study, the performance of four different shadow detection methods used in satellite imagery was evaluated for high-resolution UAV imagery collected over a California vineyard during the Grape Remote sensing Atmospheric Profile and Evapotranspiration eXperiment (GRAPEX) field campaigns. The performance of the shadow detection methods was compared and impacts of shadowed areas on the normalized difference vegetation index (NDVI) and estimated evapotranspiration (ET) using the Two-Source Energy Balance (TSEB) model are presented. The results indicated that two of the shadow detection methods, the supervised classification and index-based methods, had better performance than two other methods. Furthermore, assessment of shadowed pixels in the vine canopy led to significant differences in the calculated NDVI and ET in areas affected by shadows in the high-resolution imagery. Shadows are shown to have the greatest impact on modeled soil heat flux, while net radiation and sensible heat flux are less affected. Shadows also have an impact on the modeled Bowen ratio (ratio of sensible to latent heat) which can be used as an indicator of vine stress level.info:eu-repo/semantics/acceptedVersio

Estimation of Surface Thermal Emissivity in a Vineyard for UAV Microbolometer Thermal Cameras Using NASA HyTES Hyperspectral Thermal, and Landsat and AggieAir Optical Data

Microbolometer thermal cameras in UAVs and manned aircraft allow for the acquisition of highresolution temperature data, which, along with optical reflectance, contributes to monitoring and modeling of agricultural and natural environments. Furthermore, these temperature measurements have facilitated the development of advanced models of crop water stress and evapotranspiration in precision agriculture and heat fluxes exchanges in small river streams and corridors. Microbolometer cameras capture thermal information at blackbody or radiometric settings (narrowband emissivity equates to unity). While it is customary that the modeler uses assumed emissivity values (e.g. 0.99– 0.96 for agricultural and environmental settings); some applications (e.g. Vegetation Health Index), and complex models such as energy balance-based models (e.g. evapotranspiration) could benefit from spatial estimates of surface emissivity for true or kinetic temperature mapping. In that regard, this work presents an analysis of the spectral characteristics of a microbolometer camera with regard to emissivity, along with a methodology to infer thermal emissivity spatially based on the spectral characteristics of the microbolometer camera. For this work, the MODIS UCBS Emissivity Library, NASA HyTES hyperspectral emissivity, Landsat, and Utah State University AggieAir UAV surface reflectance products are employed. The methodology is applied to a commercial vineyard agricultural setting located in Lodi, California, where HyTES, Landsat, and AggieAir UAV spatial data were collected in the 2014 growing season. Assessment of the microbolometer spectral response with regards to emissivity and emissivity modeling performance for the area of study are presented and discussed

To What Extent Does the Eddy Covariance Footprint Cutoff Influence the Estimation of Surface Energy Fluxes Using Two Source Energy Balance Model and High-Resolution Imagery in Commercial Vineyards?

Validation of surface energy fluxes from remote sensing sources is performed using instantaneous field measurements obtained from eddy covariance (EC) instrumentation. An eddy covariance measurement is characterized by a footprint function / weighted area function that describes the mathematical relationship between the spatial distribution of surface flux sources and their corresponding magnitude. The orientation and size of each flux footprint / source area depends on the micro-meteorological conditions at the site as measured by the EC towers, including turbulence fluxes, friction velocity (ustar), and wind speed, all of which influence the dimensions and orientation of the footprint. The total statistical weight of the footprint is equal to unity. However, due to the large size of the source area / footprint, a statistical weight cutoff of less than one is considered, ranging between 0.85 and 0.95, to ensure that the footprint model is located inside the study area. This results in a degree of uncertainty when comparing the modeled fluxes from remote sensing energy models (i.e., TSEB2T) against the EC field measurements. In this research effort, the sensitivity of instantaneous and daily surface energy flux estimates to footprint weight cutoffs are evaluated using energy balance fluxes estimated with multispectral imagery acquired by AggieAir sUAS (small Unmanned Aerial Vehicle) over commercial vineyards near Lodi, California, as part of the ARS-USDA Agricultural Research Service’s Grape Remote Sensing Atmospheric Profile and Evapotranspiration eXperiment (GRAPEX) project. The instantaneous fluxes from the eddy covariance tower will be compared against instantaneous fluxes obtained from different TSEB2T aggregated footprint weights (cutoffs). The results indicate that the size, shape, and weight of pixels inside the footprint source area are strongly influenced by the cutoff values. Small cutoff values, such as 0.3 and 0.35, yielded high weights for pixels located within the footprint domain, while large cutoffs, such as 0.9 and 0.95, result in low weights. The results also indicate that the distribution of modelled LE values within the footprint source area are influenced by the cutoff values. A wide variation in LE was observed at high cutoffs, such as 0.90 and 0.95, while a low variation was observed at small cutoff values, such as 0.3. This happens due to the large number of pixel units involved inside the footprint domain when using high cutoff values, whereas a limited number of pixels are obtained at lower cutoff values

Estimation of Evapotranspiration and Energy Fluxes Using a Deep-Learning-Based High-Resolution Emissivity Model and the Two-Source Energy Balance Model with sUAS Information

Surface temperature is necessary for the estimation of energy fluxes and evapotranspiration from satellites and airborne data sources. For example, the Two-Source Energy Balance (TSEB) model uses thermal information to quantify canopy and soil temperatures as well as their respective energy balance components. While surface (also called kinematic) temperature is desirable for energy balance analysis, obtaining this temperature is not straightforward due to a lack of spatially estimated narrowband (sensor-specific) and broadband emissivities of vegetation and soil, further complicated by spectral characteristics of the UAV thermal camera. This study presents an effort to spatially model narrowband and broadband emissivities for a microbolometer thermal camera at UAV information resolution (~0.15 m) based on Landsat and NASA HyTES information using a deep learning (DL) model. The DL model is calibrated using equivalent optical Landsat / UAV spectral information to spatially estimate narrowband emissivity values of vegetation and soil in the 7–14- nm range at UAV resolution. The resulting DL narrowband emissivity values were then used to estimate broadband emissivity based on a developed narrowband-broadband emissivity relationship using the MODIS UCSB Emissivity Library database. The narrowband and broadband emissivities were incorporated into the TSEB model to determine their impact on the estimation of instantaneous energy balance components against ground measurements. The proposed effort was applied to information collected by the Utah State University AggieAir small Unmanned Aerial Systems (sUAS) Program as part of the ARS-USDA GRAPEX Project (Grape Remote sensing Atmospheric Profile and Evapotranspiration eXperiment) over a vineyard located in Lodi, California. A comparison of resulting energy balance component estimates, with and without the inclusion of high-resolution narrowband and broadband emissivities, against eddy covariance (EC) measurements under different scenarios are presented and discussed

Incorporation of Unmanned Aerial Vehicle (UAV) Point Cloud Products into Remote Sensing Evapotranspiration Models

In recent years, the deployment of satellites and unmanned aerial vehicles (UAVs) has led to production of enormous amounts of data and to novel data processing and analysis techniques for monitoring crop conditions. One overlooked data source amid these efforts, however, is incorporation of 3D information derived from multi-spectral imagery and photogrammetry algorithms into crop monitoring algorithms. Few studies and algorithms have taken advantage of 3D UAV information in monitoring and assessment of plant conditions. In this study, different aspects of UAV point cloud information for enhancing remote sensing evapotranspiration (ET) models, particularly the Two-Source Energy Balance Model (TSEB), over a commercial vineyard located in California are presented. Toward this end, an innovative algorithm called Vegetation Structural-Spectral Information eXtraction Algorithm (VSSIXA) has been developed. This algorithm is able to accurately estimate height, volume, surface area, and projected surface area of the plant canopy solely based on point cloud information. In addition to biomass information, it can add multi-spectral UAV information to point clouds and provide spectral-structural canopy properties. The biomass information is used to assess its relationship with in situ Leaf Area Index (LAI), which is a crucial input for ET models. In addition, instead of using nominal field values of plant parameters, spatial information of fractional cover, canopy height, and canopy width are input to the TSEB model. Therefore, the two main objectives for incorporating point cloud information into remote sensing ET models for this study are to (1) evaluate the possible improvement in the estimation of LAI and biomass parameters from point cloud information in order to create robust LAI maps at the model resolution and (2) assess the sensitivity of the TSEB model to using average/nominal values versus spatially-distributed canopy fractional cover, height, and width information derived from point cloud data. The proposed algorithm is tested on imagery from the Utah State University AggieAir sUAS Program as part of the ARS-USDA GRAPEX Project (Grape Remote sensing Atmospheric Profile and Evapotranspiration eXperiment) collected since 2014 over multiple vineyards located in California. The results indicate a robust relationship between in situ LAI measurements and estimated biomass parameters from the point cloud data, and improvement in the agreement between TSEB model output of ET with tower measurements when employing LAI and spatially-distributed canopy structure parameters derived from the point cloud data